Emergency workflow trigger

ABSTRACT

Detection of crisis situations using multiple classifiers can enable fast and efficient processing of input phrases to determine if an abatement protocol should be executed. An input phrase can be passed through a pattern matching classifier and then through a trained machine learning classifier. If neither classifier identifies a crisis, the workflow can continue as usual. However, if a crisis is identified, confirmation of the crisis situation can be sought and used to further update one or both of the classifiers. If the crisis is confirmed, emergency information and crisis management tools can be presented to the user, among other mitigating actions. If the crisis is not confirmed, a prompt can be presented to the user to discuss the trigger phrase associated with the trigger signal.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. Provisional PatentApplication No. 63/120,810 filed Dec. 3, 2020 and entitled “EMERGENCYWORKFLOW TRIGGER,” which is hereby incorporated by reference in itsentirety.

TECHNICAL FIELD

The present disclosure relates to linguistic analysis generally and morespecifically to providing emergency responses during natural languageprocessing workflows.

BACKGROUND

Natural language processing (NLP) is used in many fields to provide acomfortable and relatable interface for various purposes. When a usercommunicates to a computer system via natural language, such as throughtext input (e.g., typing a message) or audio input (e.g., speaking amessage), the computer system attempts to determine an intended meaningassociated with the received input. For example, in the field of humanpsychology, artificial intelligence systems can use NLP to interact withthe user and provide helpful tools, commentary, or other conversationwith the user in a natural and comfortable fashion.

SUMMARY

The term embodiment and like terms are intended to refer broadly to allof the subject matter of this disclosure and the claims below.Statements containing these terms should be understood not to limit thesubject matter described herein or to limit the meaning or scope of theclaims below. Embodiments of the present disclosure covered herein aredefined by the claims below, supplemented by this summary. This summaryis a high-level overview of various aspects of the disclosure andintroduces some of the concepts that are further described in theDetailed Description section below. This summary is not intended toidentify key or essential features of the claimed subject matter, nor isit intended to be used in isolation to determine the scope of theclaimed subject matter. The subject matter should be understood byreference to appropriate portions of the entire specification of thisdisclosure, any or all drawings and each claim.

Embodiments of the present disclosure include a computer-implementedmethod, comprising: receiving an input phrase associated with a crisissituation; applying a first classifier on the input phrase to determinethat the input phrase does not trigger the first classifier; applying asecond classifier on the input phrase to determine that the input phrasetriggers the second classifier, wherein the first classifier is one of apattern matching classifier and a trained machine learning classifier,and wherein the second classifier is the other of the pattern matchingclassifier and the trained machine learning classifier; generating atrigger signal in response to determining that the input phrase triggersthe second classifier; and executing an abatement protocol in responseto generation of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.

In some cases, executing the abatement protocol includes i) accessingand presenting a text string containing emergency contact information,ii) accessing and presenting a link configured to facilitate anemergency contact connection upon actuation; iii) sending a signal toautomatically facilitate an emergency contact connection; iv) initiatinga patient health questionnaire protocol; v) generating a prompt forselecting a crisis management tool; vi) automatically initiating acrisis management tool; or vii) any combination of i-vi. In some cases,the method further comprises presenting a crisis confirmation prompt inresponse to determining that the input phrase triggers the secondclassifier; and receiving a confirmation response in response topresenting the confirmation prompt, the confirmation response includinga confirmation of the crisis situation or a denial of the crisissituation, wherein executing the abatement protocol occurs in responseto receiving the confirmation of the crisis situation.

In some cases, the method further comprises identifying a trigger phrasein response to determining that the input phrase triggers the secondclassifier, wherein the trigger phrase includes a portion of the inputphrase that matched a regular expression when the second classifier isthe pattern matching classifier, and wherein the trigger phrase includesthe input phrase when the second classifier is the trained machinelearning classifier; and presenting a denial prompt in response toreceiving the denial of the crisis situation, wherein presenting thedenial prompt includes using the trigger phrase. In some cases, themethod further comprises logging the confirmation response inassociation with the trigger phrase; and updating at least one of thefirst classifier and the second classifier using the logged confirmationresponse and the trigger phrase. In some cases, executing the abatementprotocol includes scheduling a follow-up contact for a future time,wherein the method further comprises presenting a follow-up message atthe future time, and wherein presenting the follow-up message includespresenting the trigger phrase.

In some cases, the method further comprises determining a crisis scoreusing the input phrase, wherein generating the trigger signal is furtherbased on a determination that the crisis score is outside of a thresholdrange. In some cases, the method further comprises determining a crisisscore using the input phrase; accessing one or more historical crisisscores; determining a crisis score trend using the crisis score and theone or more historical crisis scores; and determining a future crisisscore using the crisis score and the crisis score trend, whereingenerating the trigger signal is further based on a determination thatthe future crisis score is outside of a threshold range. In some cases,the first classifier is the pattern matching classifier and the secondclassifier is the trained machine learning classifier. In some cases,the trained machine learning classifier is trained using a set oftraining input phrases that were determined to not trigger the patternmatching classifier. In some cases, the trained machine learningclassifier is trained using a set of training input phrases including afirst subset of training input phrases associated with crisis situationsand a second subset of training input phrases not associated with crisissituations.

Embodiments of the present disclosure include a system comprising one ormore data processors; and a non-transitory computer-readable storagemedium containing instructions which, when executed on the one or moredata processors, cause the one or more data processors to perform theabove method.

Embodiments of the present disclosure include a computer-program producttangibly embodied in a non-transitory machine-readable storage medium,including instructions configured to cause a data processing apparatusto perform the above method.

BRIEF DESCRIPTION OF THE DRAWINGS

The specification makes reference to the following appended figures, inwhich use of like reference numerals in different figures is intended toillustrate like or analogous components.

FIG. 1 is a schematic diagram depicting a computing environmentaccording to certain aspects of the present disclosure.

FIG. 2 is a flowchart depicting a process for identifying a crisissituation according to certain aspects of the present disclosure.

FIG. 3 is a flowchart depicting a process for confirming and respondingto a crisis situation according to certain aspects of the presentdisclosure.

FIG. 4 is a flowchart depicting a process for quantifying andidentifying a crisis situation according to certain aspects of thepresent disclosure.

FIG. 5 is a block diagram depicting an example system architecture forimplementing certain features and processes of the present disclosure

DETAILED DESCRIPTION

Certain aspects and features of the present disclosure relate to anatural language processing (NLP) system capable of detecting crisissituations through the use of multiple classifiers. The use of multipleclassifiers can enable fast and efficient processing of input phrases todetermine if an abatement protocol should be executed. An input phrasecan be passed through a pattern matching classifier and then through atrained machine learning classifier. If neither classifier identifies acrisis, the workflow can continue as usual. However, if a crisis isidentified, confirmation of the crisis situation can be sought and usedto further update one or both of the classifiers. If the crisis isconfirmed, emergency information and crisis management tools can bepresented to the user, among other mitigating actions. If the crisis isnot confirmed, a prompt can be presented to the user to discuss thetrigger phrase associated with the trigger signal.

Aspects and features of the present disclosure can be used in variousenvironments and for various purposes. In some cases, aspects andfeatures of the present disclosure enable automatic detection of crisissituations based on received input, which can be especially useful inhuman-computer interactions, such as artificial intelligence chat-basedtools, commonly known as chatbots. In some cases, the present disclosurecan be especially useful when used with chatbots used for therapy, suchas the monitoring and/or treatment of mental health disorders. In suchcases, it can be especially important to identify whether or not anindividual interacting with the chatbot is undergoing a crisissituation. If a crisis situation is detected, the chatbot (optionallyafter confirmation from the individual) can execute a protocol thatincludes taking actions to help mitigate the crisis situation orotherwise aid the individual in managing the crisis situation. As usedherein with respect to a crisis situation, the term “mitigate” isinclusive of both affecting the crisis situation itself, as well asaffecting the individual to better manage or undergo the crisissituation. In cases where a chatbot is used, inputs are normallyreceived in the form of text selected from a list or entered into afield, although that need not always be the case. For example, in somecases, individuals can speak or dictate to the chatbot.

While described with reference to a chatbot in many places herein,certain aspects and features of the present disclosure can be used forother purposes, such as to detect crisis situations in otherhuman-computer interactions, such as personal journaling applications,voice-activated devices (e.g., voice-activated assistants),voice-activated telephonic prompts (e.g., a voice-activated menu servicefor a telephone line), and the like. Certain aspects and features of thepresent disclosure can also be used to detect crisis situations inhuman-human interactions, such as text-based or audio-basedcommunications between individuals locally or remotely. For example, atherapist treating a patient may make use of a system that automaticallydetects the patient's interactions to identify whether or not a crisissituation may be occurring, notifying the therapist or making a logentry when a crisis situation is detected.

While many applications of the aspects and features of the presentdisclosure are especially useful for automatic and dynamic crisisdetection, such as realtime crisis detection, that need not always bethe case. In some cases, an input phrase can be stored for laterprocessing, only being processed at a later time to identify whether ornot a crisis situation occurred when the input phrase was supplied. Insuch cases, various actions, such as execution of an abatement protocol,can still be performed.

An individual can provide an input phrase, such as via text entry in achat box, by selecting a text entry, by speaking words aloud, orotherwise. The term input phrase is inclusive of any suitable collectionof inputs that conveys linguistic meaning, such as a single word,multiple words, a single sentence, multiple sentences, or even symbols(e.g., emoticons and the like). The input phrase can be pre-processed asnecessary to achieve a standard type of input for the classifiers (e.g.,an audio signal can be processed through a speech-to-text processor togenerate corresponding text).

The input phrase can be processed by a pattern matching classifier and atrained machine learning classifier (optionally sequentially in thatorder) to determine if at least one classifier returns a trigger signal.If at least one of (or optionally both of) the classifiers returns atrigger signal, a trigger signal can be sent to initiate a crisisworkflow, optionally including a trigger phrase (e.g., the input phraseor a portion of the input phrase that caused the classifier to trigger)and/or other information.

The crisis workflow can request confirmation from the individual that acrisis situation does exist. The confirmation can be stored, optionallyalong with the trigger signal and/or other information, and later usedto improve future classification. If the crisis situation is confirmed,an abatement protocol can be executed, which can include i) accessingand presenting a text string containing emergency contact information,ii) accessing and presenting a link configured to facilitate anemergency contact connection upon actuation; iii) sending a signal toautomatically facilitate an emergency contact connection; iv) initiatinga patient health questionnaire protocol; v) generating a prompt forselecting a crisis management tool; vi) automatically initiating acrisis management tool; or vii) any combination of i-vi. In some cases,a follow-up can be automatically or manually scheduled, which can thenbe used to present a follow-up message some time in the future to checkin on the individual.

In some cases, if the crisis situation is denied, a denial prompt can bepresent to obtain more information about why the detected crisissituation was denied, and/or provide information that may be useful toone who denied the crisis situation. In some cases, presenting a denialprompt can include using the trigger phrase, such as presenting thetrigger phrase and asking the individual why the trigger phrase is notindicative of a crisis situation, asking why the individual provided thetrigger phrase, and/or what the trigger phrase evoked in the individual.

The use of both a pattern matching classifier and a trained machinelearning classifier has been found to be especially useful forclassification of crisis situations. In the detection of crisissituations, it can be important to maximize different aspects of theperformance of the classifier(s). For example, it can be detrimental tofail to classify too many input phrases that were actually crisissituations, as these individuals may not receive beneficial andimportant help. Additionally, it can be detrimental to falsely classifytoo many non-crisis input phrases as crisis situations, as it can annoyusers and discourage individuals from making use of the underlyingsystems (e.g., the chatbot). Depending on the needs associated with theparticular use case, it can be important to maximize or minimizedifferent operational statistics of one or both classifiers.Additionally, while use of a trained machine learning classifier (e.g.,a trained neural network classifier) is useful for detecting crisissituations that would not otherwise be captured by the pattern matchingclassifier, the use of the pattern matching classifier is useful todetect certain words, collections of words, or other patterns that mayalways be indicative of a crisis situation or that may be stronglyindicative of a crisis situation.

A pattern matching classifier can be any suitable classifier that usesmatching criteria to identify whether or not the input phrase fits thematching criteria. In some cases, simple pattern matching classifierscan include find or search functions that simply determine whether ornot a certain word or collection of words is present in the inputphrase. In some cases, more complex pattern matching classifiers can beused, such as a regular expression (RegEx) classifier. In such cases,the matching criteria can be the regular expressions used to determinewhether or not the input phrase first matches the set of regularexpressions. For example, a regular expression can be“\bkill\W+(?:\w+\W+){1,6}?(\w+){0,}self” and can search for instances ofwords containing “self” within six words of the word “kill.” A set ofregular expressions can include one or more inclusionary regularexpressions and/or one or more exclusionary regular expressions. In somecases, use of a RegEx classifier has been especially useful, althoughany suitable pattern matching classifier may be used. The patternmatching classifier can be a classifier that does not use a trainedmachine learning model.

A trained machine learning classifier is any suitable classifier thatuses machine learning to train the classifier, such as using supervisedor unsupervised training. The classifier can be a machine learningmodel. Any suitable machine learning model classifier can be used, suchas a neural network classifier or deep neural network classifier. Insome cases, a recurrent neural network (RNN), optionally with a longshort-term memory (LSTM) architecture, can be used. In some cases, themachine learning model can be a transformer-based model, which canprocess a full input phrase at a time, rather than a RNN, which mayprocess individual tokens of an input phrase one-at-a-time. In somecases, the trained machine learning classifier can be a model that ispretrained for use with language, and then further trained or fine-tunedspecifically for detection of crisis situations. Examples of pretrainedtransformer-based models include a Generative Pretrained Transformer 2(GPT-2) model and a Bidirectional Encoder Representations fromTransformers (BERT) model. Training (e.g., initial training or furthertraining) and/or fine-tuning can include using training data thatincludes a set of input phrases comprising a subset of input phrasesassociated with a crisis situation and a subset of input phrases notassociated with a crisis situation. In some cases, use of a BERTclassifier has been especially useful, although any suitable trainedmachine learning classifier may be used.

When classifiers are being compared, updated, trained, and otherwiseimproved, metrics about the performance of the classifier can beobtained. Such metrics are often based on the different possibleoutcomes to the classification of an input phrase. The outcome for eachinput phrase can be i) a true positive (TP), representing a correctclassification as a crisis situation; ii) a false positive (FP),representing an incorrect classification as a crisis situation; iii) afalse negative (FN), representing an incorrect determination that nocrisis situation exists; and iv) a true negative (TN), representing acorrect determination that no crisis situation exists. In some cases,the knowledge of whether or not a classification is correct can be basedon existing training data. In some cases, however, knowledge of whetheror not a classification is correct can be based on logged confirmationresponses, as disclosed herein.

Using an example dataset of 17,030 input phrases, classifiers wereprepared and used to determine whether or not a crisis situation wasdetected. This example dataset was processed through multiple NLPsystems, each with a different classifier structure, the results ofwhich are provided in the table below. In a first example, identified as“Pattern Match Only 1,” the dataset was processed by only a RegExclassifier that included a set of inclusionary and exclusionary regularexpressions. In a second example, identified as “Pattern Match Only 2,”the dataset was processed by only a RegEx classifier that was similar tothe classifier of Pattern Match Only 1, except without exclusionaryregular expressions and with a more limited set of inclusionary regularexpressions. In a third example, identified as “Pattern Match+TrainedMachine Learning,” the dataset was processed by a dual classifierincluding a RegEx classifier and a BERT classifier, such as describedherein according to certain aspects and features of the presentdisclosure.

Pattern Match + Pattern Pattern Trained Machine Match Only 1 Match Only2 Learning Confusion 68 1702 56 1714 1724 46 Matrix 70 15190 14 15246 7815182 Accuracy 0.8959 0.8959 0.9927 Sensitivity 0.0384 0.0384 0.9740Specificity 0.9954 0.9954 0.9949 (Recall) Precision 0.0044 0.0044 0.1020F-Score 0.8541 0.8541 0.9927

As seen in the table above, the initial Pattern Match Only 1 classifierwas able to achieve a somewhat strong accuracy and specificity, but leftroom for improvement, especially with respect to the large number ofinstances of false positives and the overall low precision andsensitivity. Modifications were made to Pattern Match Only 1 to createthe Pattern Match Only 2 classifier, and while the Pattern Match Only 2classifier did significantly reduce the number of false negatives, itstill maintained a large number of false positives and fewer truepositives. In the Pattern Match+Trained Machine Learning classifier, thenumber of true positives increased dramatically and the number of falsepositives decreased dramatically, all with only a slight increase in thenumber of false negatives. Some trials (not depicted in the table above)included use of only a trained machine learning classifier, but it wasdetermined that a combination of a Pattern Match classifier and atrained machine learning classifier would be more beneficial. Such acombination is especially more beneficial in production environmentssince the pattern matches may improve generalization and ensure a matchagainst certain critical patterns.

Through various trials, it was determined that use of a pattern matchingclassifier (e.g., a RegEx classifier) and a trained machine learningclassifier (e.g., a BERT classifier) was beneficial, as it could provideexcellent performance. The pattern matching classifier could be adjustedto minimize the number of false positives and false negatives created,while still ensuring that certain key trigger phrases would be captured,all while the trained machine learning classifier can be used to pick upother crisis situations that would otherwise not be captured by thepattern matching classifier. Therefore, at least with reference to thefigures below, in some cases, reference to a pattern matching classifiercan be considered as reference to a RegEx classifier, and/or referenceto a trained machine learning classifier can be considered as referenceto a BERT classifier.

The pattern matching classifier and trained machine learning classifiercan be used in any suitable order, including simultaneously. However, insome cases, it can be beneficial to have the input phrase be processedby the pattern matching classifier first, before being sent to thetrained machine learning classifier. In such cases, the pattern matchingclassifier, which is often a faster operating classifier, can quicklydetermine whether or not the input phrase fits in its definition as acrisis situation. If so, the crisis workflow can be started immediately.However, if the pattern matching classifier does not classify the inputphrase as a crisis situation, that input phrase can be passed to thetrained machine learning classifier for classification. Thus, thetrained machine learning classifier may only ever receive a subset ofthe full set of input phrases, namely those input phrases that thepattern matching classifier deems are not crisis situations. In somecases, the trained machine learning classifier can be trained (e.g.,fine-tuned) based on a full set of training data. However, in somecases, the trained machine learning classifier can be trained only oninput phrases that have already been classified by the pattern matchingclassifier as not crisis situations.

In some cases, prior to attempting to detect a crisis situation from aninput phrase and/or upon detecting a crisis situation, the NLP systemcan present the user with a competence statement. The competencestatement can be an indication of the limits of the NLP system's abilityto detect crisis situations and the limits of actions that the NLPsystem may be able to take. In some cases, the competence statement canbe updated automatically using the current operational statistics of theclassifiers used to detect crisis situations. In some cases, thecompetence statement can be updated automatically based on the availableactions the NLP system can perform in response to detection of thecrisis situation. In an example, an NLP system with no ability to directemergency personnel to the user's location may indicate such limitationsin the competence statement. The competence statement can be tailored toprovide adequate notice to a user based on generally accepted ethicalprinciples of boundaries of competence, such as those promulgated by theAmerican Psychological Association or the Psychological Society ofIreland.

Aspects and features of the present disclosure provide variousimprovements to the technological process of natural languageprocessing, especially with respect to handling crisis situations.Examples of such improvements include substantial increases to accuracyand/or sensitivity without compromising specificity. Further, certainaspects and features of the present disclosure enable an administratorto maximize or minimize different operational statistics of one or moreclassifiers, allowing the administrator to customize the overallclassifying performance of the system in a new way. Additionally,certain aspects of the present disclosure, including the use of a RegExclassifier combined with a trained machine learning classifier providespecific improvements to the technological process of identifying crisissituations in natural language processing. For example, the machinelearning classifier can be trained to identify crisis situations that anindividual setting up the system might not have predicted, while theRegEx classifier can ensure certain known or predicted phrases or inputsare properly classified even if they would not trigger the machinelearning classifier.

These illustrative examples are given to introduce the reader to thegeneral subject matter discussed here and are not intended to limit thescope of the disclosed concepts. The following sections describe variousadditional features and examples with reference to the drawings in whichlike numerals indicate like elements, and directional descriptions areused to describe the illustrative embodiments but, like the illustrativeembodiments, should not be used to limit the present disclosure. Theelements included in the illustrations herein may not be drawn to scale.

FIG. 1 is a schematic diagram depicting a computing environment 100according to certain aspects of the present disclosure. The environment100 can be located in a single physical location or can be distributedabout multiple physical locations. Environment 100 can be used toimplement an NLP system, such as being used to receive input phrases,process the input phrases to determine whether or not a crisis situationexists, and take any necessary actions, such as described in furtherdetail herein. Environment 100 is an example of a suitable environmentfor implementing the NLP system, although other environments can be usedinstead.

Environment 100 can include a user device 102, a communication device108, and a server 106, although in some cases one or two of thesedevices may not be included. For example, some environments contain onlya user device 102, and some environments contain only a user device 102and a communication device 108. In some cases, multiple user devices 102can be used. In some cases, other devices can be used. When multipledevices are used, each device can be communicatively coupled together,such as via a direct connection (e.g., a wired connection, such as auniversal serial bus (USB) connection, or a wireless connection, such asa Bluetooth connection) or via network 110. Network 110 can be anysuitable network, such as a local area network, a wide area network, acloud, or the Internet.

An individual can provide an input phrase to the NLP system, which canthen process the input phrase to determine whether or not a crisissituation exists. The NLP system can be implemented on a single device(e.g., on user device 102) or can be implemented across multiple devices(e.g., any combination of user device 102, communication device 108, andsever(s) 106).

The NLP system can process the input phrase by applying a patternmatching classifier and a trained machine learning classifier. Eachclassifier can have classifier parameters that define how the classifierfunctions. In an example, the pattern matching classifier's classifierparameters can include matching criteria (e.g., a set of regularexpressions containing one or more regular expressions), which can beconfigured to trigger whenever certain trigger phrases (e.g., triggeringwords, triggering collections of words, triggering symbols, triggeringpatterns of words, or the like) are present (e.g., matched) in the inputphrase. In another example, the trained machine learning classifier'sclassifier parameters can include a model (e.g., a transformer model)that has already been trained for the purpose of identifying crisissituations.

When a crisis situation is detected, the NLP system can requestconfirmation from the individual (e.g., a user who provided the inputphrase) or from another individual (e.g., a caregiver of medicalprofessional treating a patient who provided the input phrase), who canprovide a confirmation response that is either a confirmation or adenial of the crisis situation.

The NLP system can execute an abatement protocol, such as if the crisissituation is confirmed, which can involve communicating information tothe individual, facilitating communication between the individual andanother, providing one or more prompts to the user, providing one ormore crisis management tools to the user, or otherwise engaging theuser. In some cases, the NLP system can present a denial prompt to theuser when the crisis situation is denied. The denial prompt can be usedto discuss the trigger phrase with the user. Requesting confirmation,executing an abatement protocol, and/or presenting the denial prompt canbe performed using any suitable component or combination of componentsof environment 100, such as the user device 102, the communicationdevice 108, and server(s) 106.

A user device 102 can act as a primary mode of interaction for one ormore individuals to provide input phrases, receive prompts andinformation, and otherwise interact with the NLP system. Examples ofuser device 102 include any suitable computing device, such as apersonal computer, a smartphone, a tablet computer, a smartwatch, acomputerized audio recorder, or the like. User device 102 can beoperatively coupled to storage 104 to store data associated withapplications and processes running on the user device 102. User device102 can include any combination of input/output (I/O) devices that maybe suitable for interacting with the NLP system, such as a keyboard, amouse, a display, a touchscreen, a microphone, a speaker, an inertialmeasurement unit (IMU), a haptic feedback device, or other such devices.

A communication device 108 can be used to bridge the NLP system with aseparate communication system, such as a telephone system. In somecases, communication device 108 can be used to provide input phrases tothe NLP system. In some cases, communication device 108 can be used inthe execution of an abatement protocol, such as to facilitate initiationof a communication between the user device 102 and an emergency serviceprovider (e.g., a crisis support hotline). In some cases, however, userdevice 102 can act as a communication device 108.

One or more servers 106 can be used to process input phrases, such asinput phrases received from user device 102 via network 110, and takefurther action (e.g., execute an abatement protocol or present a denialprompt). In some cases, server(s) 106 can receive an indication that acrisis situation has been detected (e.g., from a user device 102), thenfacilitate taking further action (e.g., by transmitting appropriateemergency information to user device 102 or by facilitating an emergencycontact connection, such as via communication device 108).

In some cases, server(s) 106 can be used to receive information that canbe used to update one or both of the classifiers. For example, server(s)106 can receive a confirmation response and optionally an associatedclassifier (e.g., the triggering classifier that prompted theconfirmation prompt) and/or an associated trigger phrase. Thisinformation can be used to automatically or manually update classifierparameters and improve one or both classifiers. For example, if thepattern matching classifier regularly identifies a particular phrase asindicative of a crisis situation, but individuals regularly provideconfirmation responses that are denials, the matching criteria of thepattern matching classifier can be modified to no longer trigger basedon the particular phrase. In another example, confirmation responsesassociated with a trigger phrase (e.g., the input phrase) can be used toimprove training of the trained machine learning classifier's model.Even in cases where no trigger phrase is associated with theconfirmation response, a confirmation response associated with thetriggering classifier can help improve calculation of the particularclassifier's operational statistics, including the classifier'saccuracy, sensitivity, specificity, precision, and/or F-score.Calculation of such operational statistics can inform the need forupdates or further training.

In some cases, classifying an input phrase can occur directly on theuser device 102, such as using classifiers and/or classifier parametersstored in storage 104. In some cases, classifying an input phrase canoccur at server(s) 106, with the user device 102 providing the inputphrase to the server(s) 106. In some cases, classifying an input phrasecan be split, with one classifier (e.g., the pattern matchingclassifier) running on the user device 102, and the second classifier(e.g., the trained machine learning classifier) running on the server(s)106. In such cases, the user device 102 may be able to handleclassification locally if the first classifier determines that the inputphrase is associated with a crisis situation.

FIG. 2 is a flowchart depicting a process 200 for identifying a crisissituation according to certain aspects of the present disclosure.Process 200 can be performed by an NLP system in any suitableenvironment, such as environment 100 of FIG. 1. Process 200 can berepeated any number of times, such as once for each time the userprovides an input phrase.

At block 202, an input phrase can be received. The input phrase can be asingle character (e.g., a single symbol or emoticon), multiplecharacters, a single word, multiple words, an ordered string of words, asentence, multiple sentences, or the like. In some cases, receiving aninput phrase at block 202 can include concatenating multiple textentries, such as if an individual types and sends multiple messages in arow using a chatbot interface.

At optional block 204, the input context can be checked and/or acrisis-disable switch can be checked. Checking input context can be auseful tool to avoid invoking the crisis-checking classifiers when theymay not be needed, such as if an individual is intentionally using inputphrases that would normally trigger a finding of a crisis situation asan example or in a tool where the individual is instructed to providecertain phrases that may otherwise normally trigger a finding of acrisis situation. Checking input context can include identifying acontext associated with receiving the input, such as identifying aparticular tool or application being used to provide the input, orphrases that otherwise indicate the input phrase should not invoke thecrisis-checking classifiers. Depending on the input context, the NLPsystem can either proceed with crisis checking at block 208, or can skipthe crisis checking and instead continue the normal workflow at block206.

In some cases, at optional block 204, a crisis-disable switch can bechecked. The crisis-disable switch can be a physical or digital switchor setting that permits input phrases to be received without beingchecked by one, some, or all the crisis-checking classifiers. In somecases, the crisis-disable switch can be automatically triggered, such asbased on input context or in response to a denial of a crisis situationassociated with a recent input phrase triggering a crisis-checkingclassifier.

For a crisis-disable switch that is meant to disable allcrisis-checking, when that crisis-disable switch is enabled, thereceived input phrase from block 202 will be processed according to thenormal workflow at block 206. However, when the crisis-disable switch isdisabled, the received input phrase from block 202 will be processednormally by the crisis-checking classifiers at block 208. In some cases,the crisis-disable switch can be designed to disable at least one, butfewer than all of the crisis-checking classifiers. For example, thecrisis-disable switch can be designed to disable the trained machinelearning classifier, thus only using the pattern matching classifier toperform crisis-checking. In such an example, crisis-matching can beturned off for everything except for certain critical input phrases thatwould be matched by the pattern matching classifier. In some cases, whenthe crisis-disable switch is enabled, alternate classifier parameterscan be used by one or more of the classifiers. For example, when thecrisis-disable switch is enabled, the pattern matching classifier mayuse alternate classifier parameters that only match for a relativelysmall set of input phrases as compared to the classifier parameters whenthe crisis-disable switch is not enabled.

Continuing the normal workflow at block 206 can include processing theinput phrase as it would otherwise be processed without crisis checking.For example, in the case of an artificial intelligence chatbot,continuing normal workflow may include processing the input phrasethrough one or more additional classifiers, and otherwise processing theinput phrase to determine a response to give, then providing thatresponse. As part of or after completing the normal workflow at block206, process 200 can repeat whenever another input phrase is received ata new instance of block 202.

In some cases, continuing the normal workflow at block 206 can includeaccessing memory to determine if an immediately prior conversation wasassociated with a past crisis situation or associated with interactionsimmediately following such a past crisis situation (e.g., interactionsin response to or during execution of the abatement protocol). If theimmediately prior conversation was indeed associated with a past crisissituation or with interactions immediately following such a past crisissituation, the NLP system can present a prompt to the user that isrelated to that past crisis situation (e.g., a request to see how theindividual is feeling now that time has elapsed since the crisissituation or a request to see how the individual ended up handling thecrisis situation).

At block 208, the input phrase can be processed by the crisis-checkingclassifiers. When input context is checked and/or a crisis-disableswitch is checked at block 204, the input phrase from block 202 may onlybe processed by the crisis-checking classifier at block 208 if it isdetermined at block 204 that the crisis-checking classifier should beused. However, in some cases, the input phrase received at block 202 isautomatically processed by the crisis-checking classifiers at block 208.In some cases, the input phrase received at block 202 can be processedby the crisis-checking classifiers at block 208 before it is processedby any other classifiers associated with generating a response to theinput phrase.

At block 208, the input phrase can be classified by two classifiers.While depicted in a particular sequential order in FIG. 2, the twoclassifiers can be applied in other orders or simultaneously. At block21, a pattern matching classifier is applied to the input phrase fromblock 202. Applying the pattern matching classifier at block 210 caninclude processing the input phrase using the pattern matchingclassifier to determine whether or not the input phrase is indicative ofa crisis situation. Applying the pattern matching classifier at block210 can include searching the input phrases using a set of matchingcriteria, which can include one or more criterion. In some cases,applying the pattern matching classifier at block 210 can includeaccessing the matching criteria, such as from a local memory or via theInternet. Applying the pattern matching classifier at block 210 canresult in the pattern matching classifier triggering or not triggering.If the input phrase matches any inclusionary matching criteria, takingin account exclusionary matching criteria, if any, the classifier candeem the input phrase as being indicative of a crisis situation, and canbe triggered. If the pattern matching classifier does not trigger whenprocessing the input phrase (e.g., the input phrase does not match or isexcluded based on the matching criteria), the input phrase can beoutputted to the trained machine learning classifier or the trainedmachine learning classifier can be otherwise signaled to startprocessing the input phrase.

At block 212, the trained machine learning classifier can be applied tothe input phrase as received from the pattern matching classifier fromblock 210. Applying the trained machine learning classifier at block 210can include processing the input phrase using the trained machinelearning classifier to determine whether or not the input phrase isindicative of a crisis situation. Applying the trained machine learningclassifier at block 212 can include accessing a model associated withthe trained machine learning classifier and applying the input phrase tothe model. In some cases, the model can be accessed from a local memoryor via the Internet. Applying the trained machine learning classifier atblock 210 can result in the trained machine learning classifiertriggering or not triggering. If the results of the trained machinelearning classifier indicate the input phrase is indicative of a crisissituation, the trained machine learning classifier will trigger. If theresults of the trained machine learning classifier indicate that theinput phrase is not indicative of a crisis situation, the input phrasecan be passed to the normal workflow at block 206 or the trained machinelearning classifier can otherwise signal to the NLP system to continueprocessing the input phrase under the normal workflow at block 206. Thetrained machine learning classifier can be trained using training dataincluding a set of input phrases, the set of input phrases including asubset of input phrases associated with a crisis situation and a subsetof input phrases not associated with a crisis situation.

In some optional cases, if either the pattern matching classifier fromblock 210 or the trained machine learning classifier from block 212 aretriggered, the trigger phrase can be identified at block 214. A triggerphrase can be a portion of or an entirety of the input phrase from block202. In cases where the pattern matching classifier is triggered atblock 210, identifying the trigger phrase can include identifying, asthe trigger phrase, either the entire input phrase or the portion of theinput phrase that matched the matching criteria of the pattern matchingclassifier. In cases where the trained machine learning classifier istriggered at block 212, identifying the trigger phrase can includeidentifying the entire input phrase as the trigger phrase. In somecases, identifying the trigger phrase associated with triggering of atrained machine learning classifier can include determining contributionvalues for portions of the input phrase and selecting as the triggerphrase the portions of the input phrase having contribution values overa threshold (e.g., a static or dynamic threshold).

In response to either the pattern matching classifier from block 210 orthe trained machine learning classifier from block 212 being triggered,the crisis workflow can be performed at block 216. Performing the crisisworkflow can include optionally confirming the crisis situation,executing an abatement protocol, presenting any denial prompts, orotherwise interacting with the individual in association with the crisissituation. In some cases, performing the crisis workflow at block 216can include making use of the trigger phrase identified at block 214. Insome cases, performing the crisis workflow at block 216 can includemaking use of an indication of which classifier was triggered.

Process 200 is depicted with a certain arrangement of blocks, however inother cases, these blocks can be performed in different orders, withadditional blocks, and/or some blocks removed. For example, in somecases, the input phrase can be first provided to a trained machinelearning classifier before being provided to a pattern matchingclassifier.

FIG. 3 is a flowchart depicting a process 300 for confirming andresponding to a crisis situation according to certain aspects of thepresent disclosure. Process 300 can be performed by an NLP system in anysuitable environment, such as environment 100 of FIG. 1. Process 300 canrepresent performing a crisis workflow, such as performed with respectto block 216 of FIG. 2.

At block 302, a trigger signal can be received. The trigger signal canbe generated by or in response to a classifier determining that aparticular input phrase is indicative of a crisis situation. In somecases, receiving the trigger signal at block 302 can include receiving atrigger phrase (e.g., a trigger phrase as identified at block 214 ofFIG. 2). In some cases, receiving the trigger signal at block 302 caninclude receiving a classifier identification. The classifieridentification can be information usable to identify which classifier orwhich type of classifier was triggered.

At block 304, a confirmation prompt is presented. Presentation of theconfirmation prompt can include asking the individual if they areundergoing a crisis situation. In some cases, the confirmation promptcan be presented to someone other than the individual who provided theinput phrase, such as if a caregiver or therapist is interacting withthe individual, in which case the caregiver or therapist may be providedwith a confirmation prompt indicating that the individual may besuffering from a crisis situation.

In some cases, presenting the confirmation prompt at block 304 caninclude presenting an explanation that the crisis workflow was triggeredand/or presenting the trigger phrase or the input phrase. In an exampleof an input phrase of “I want to kill myself,” the NLP system mayidentify the phrase as matching the matching criteria of the patternmatching classifier with a trigger phrase of “want to kill,” then theNLP system can present a confirmation prompt that includes “My crisissystems have been triggered. This is because I've recognized “want tokill” as an emergency. Is this the case? Are you in crisis?” Otherexamples can be used. In some cases, presenting the confirmation promptat block 304 can include presenting a set of options from which theindividual may select a response (e.g., “Yes” or “No”). In some cases,presenting the confirmation prompt at block 304 can include requestingconfirmation in an open format (e.g., allowing a user to type in theirown response rather than selecting from a set of options).

At block 306, a confirmation response can be received. The confirmationresponse is received in response to the confirmation prompt. In somecases, the confirmation response can be one of a set of options providedat block 304. In some cases, the confirmation response can be initiallyprovided in the form of an input phrase (e.g., text input), which theNLP system can analyze to classify as appropriate (e.g., a confirmationof the crisis situation or a denial of the crisis situation, oroptionally a request for more information). Generally, a confirmationresponse will be indicative of a confirmation of the crisis situation ora denial of the crisis situation, although that need not always be thecase.

In some optional cases, depending on the input phrase used, block 304can be skipped and the trigger signal itself can be considered aconfirmation response that confirms the crisis situation. For example,an input phrase that explicitly states the individual is in a crisis orexplicitly requests crisis aid, the NLP system may automatically assumethat the input phrase itself is a confirmation response that confirmsthe existence of a crisis situation.

At block 308, the confirmation response can be logged in associated withthe trigger signal. Logging the confirmation response in associationwith the trigger signal can include transmitting and/or storing dataindicative of the confirmation response in association with the triggersignal. The trigger signal can include an indication of the inputphrase, an indication of the trigger phrase, an indication of why thetrigger signal was generated (e.g., which classifier was triggered), orother such information. Logging the confirmation response in associationwith the trigger signal can include logging the confirmation response inassociation with at least a portion of the trigger signal. For example,in some cases, the confirmation response may be logged with just thetrigger phrase. In another example, the confirmation response may belogged with the input phrase and an indication of which classifier wastriggered. In another example, the confirmation response may be loggedwith the input phrase and the trigger phrase. In such an example,knowledge of the input phrase and the trigger phrase may be usable todiscern which classifier was triggered in cases where the trainedmachine learning classifier always uses the entire trigger phrase as theinput phrase and the pattern matching classifier always uses a portionof the trigger phrase as the input phrase.

At block 310, one or both of the classifiers can be updated using thelogged confirmation response. In some cases, updating a classifier usingthe logged confirmation response can include preparing updatedperformance statistics associated with the classifier that include thelogged confirmation response. Such updated performance statistics can beused to inform an update to the classifier. In some cases, updating aclassifier using the logged confirmation response can include using thelogged confirmation response as training data (e.g., as a data point ina larger set of training data). Using the logged confirmation responseas training data can include adding the logged confirmation response toan existing set of training data, then using that set of training datato initially train a machine learning model (e.g., a model of thetrained machine learning classifier) or to update (e.g., fine-tune)training of an existing machine learning model (e.g., an existing modelof the trained machine learning classifier, such as a pretrained model).

When the NLP system assumes or knows that a crisis situation exists(e.g., via receiving a confirmation of the crisis situation at block306), the process 300 can continue at block 312 with executing anabatement protocol. Executing the abatement protocol can includeperforming or initiating the performance of one or more actions that aredesigned or selected to help mitigate the crisis situation or otherwiseaid the individual in managing the crisis situation. Such actions can beknown as mitigation actions. Executing the abatement protocol caninclude performing actions in any suitable order, includingsimultaneously, as appropriate. In some cases, executing the abatementprotocol at block 312 can include accessing preset data (e.g., textstrings, such as phrases; computer links, such as hyperlinks; images; orother content) and presenting that preset data (e.g., sending the presetdata such that the preset data, when received, is displayed on a displaydevice). In some cases, the preset data can include comforting orsympathetic phrases, images, or other content.

In some cases, executing an abatement protocol at block 312 can includepresenting emergency information at block 314. Presenting emergencyinformation can include accessing data associated with the emergencyinformation (e.g., a text string containing emergency contactinformation) and sending the data such that the data, when received, isdisplayed on a display device or otherwise provided to the user in acomprehensible fashion (e.g., via text printed by printer or audiosignals generated by a speaker). Presenting emergency information caninclude presenting preset emergency information, such as a preset listof emergency contact phone numbers or websites, preset instructions formanaging the crisis situation, or other such emergency information. Insome cases, presenting emergency information at block 314 can includeaccessing emergency information, such as from memory or from theInternet (e.g., a server on the Internet). In some cases, accessingemergency information can include providing information associated withthe individual and/or information associated with the trigger signal,which information can be used to obtain relevant emergency informationthat can then be presented. In an example, a geolocation of theindividual (e.g., based on a geolocation of a user device, a presetlocation, or one or more sensors of a user device) can be used to obtainemergency information associated with that geolocation (e.g., emergencyinformation for a particular county, city, state, country, or the like).In another example, information about the trigger signal can be used toselect appropriate emergency information suitable for the type of crisissituation at hand.

In some cases, executing an abatement protocol at block 312 can includefacilitating an emergency contact connection at block 316. Facilitatingan emergency contact connection can include presenting an option orprompt to make an emergency contact connection and/or automaticallymaking an emergency contact connection. For example, in some cases,facilitating an emergency contact connection can include accessing andpresenting a link configured to begin initiation of an emergency contactconnection upon actuation. Beginning initiation of an emergency contactconnection can include taking initial step(s) to start the connection(e.g., opening a telephone app on a smartphone and prepopulating thedial field with an emergency contact phone number without starting thephone call) or can include taking all step(s) to start the connection(e.g., opening a telephone app on a smartphone and automatically dialingthe emergency contact phone number). Such a link can be a link to begininitiation of a telephone call, a link to begin initiation of avideoconference, a link to begin initiation of a text chat, a link tobegin initiation of a text message or similar message, or other suchlinks. In another example, in some cases, facilitating an emergencycontact connection can include sending a signal to automatically begininitiation of an emergency contact connection (e.g., automatically startup a telephone app and prepopulate the given phone number withoutstarting the call or automatically start up a telephone app and call thegiven phone number). Such a signal can include a signal to automaticallybegin initiation of a phone call, a signal to automatically begininitiation of a videoconference, a signal to automatically begininitiation of a text chat, a signal to automatically begin initiation ofa text message, or the like. In some cases, actuation of a link to begininitiation of a given action can result in sending the signal toautomatically begin initiation of that action. In some cases, beginninginitiation of an action can be performed by the user's device or anotherdevice associated with the user (e.g., another device owned by the userand/or assigned to an account or unique identifier of the user). In somecases, however, beginning initiation of an action can be performed by aremote device. For example, in some cases, beginning initiation of aphone call can include sending a command to a remote system to place acall to the user's phone number.

An emergency contact connection can be an audio connection (e.g., atelephone call), a video connection (e.g., a videoconference call), atext-based connection (e.g., a text message or chat messagecommunication), or other such connections. In some cases, facilitatingthe emergency contact connection can include making an emergency contactconnection using the emergency information presented at block 314,although that need not always be the case. In some cases, facilitatingthe emergency contact connection can include beginning to initiate orautomatically initiating an emergency contact connection with a presetparty (e.g., a family member or caregiver previously selected by theindividual).

In some cases, executing an abatement protocol at block 312 can includeoffering or engaging a crisis management tool at block 318. Offering acrisis management tool can include generating a prompt that can bepresented to a user, the response to which can be used to select one ormore crisis management tools and optionally initiate the selected crisismanagement tool. Engaging a crisis management tool can includeautomatically initiating the crisis management tool. A crisis managementtool can be a workflow of prompts and/or comments used to engage theindividual in a fashion that may help mitigate the crisis situation. Inan example, a crisis management tool can be a list of actions or tasksto be accomplished, which can be selected to help mitigate the crisissituation. In another example, a crisis management tool can be a seriesof questions that can help the individual identify a cause of the crisissituation and hopefully better manage the cause of the crisis situation.In some cases, one or more crisis management tools are offered at block318, allowing the individual to select a particular crisis managementtool to use or optionally decline the crisis management tool(s). In somecases, a crisis management tool can be automatically engaged at block318.

In some cases, executing an abatement protocol at block 312 can includeperforming a patient health questionnaire (PHQ) at block 320. Performinga PHQ can include initiating a PHQ protocol to access and present one ormore questions associated with the PHQ. In some cases, the PHQ protocolcan include selecting a particular PHQ to be performed from a set ofPHQs. The PHQ protocol can include stored and/or transmitting resultdata. The result data of the PHQ protocol can include individual answersto questions, individual or cumulative scores, and/or other metricsassociated with performance of the PHQ (e.g., reaction time to answer aquestions, time of day, and the like). In some cases, results (e.g.,result data) from performing a PHQ during a crisis situation can be auseful metric to have. Performing the PHQ can include storing theresponses, optionally in association with an indicator that the PHQ wastaken during a crisis situation. Any suitable PHQ can be used, althoughuse of the Patient Health Questionnaire-2 (PHQ-2) can be especiallyuseful for certain aspects of the present disclosure, as it providesuseful information while remaining short and easy to answer, allowingother actions of the abatement protocol to be quickly performed.Performing the PHQ-2 can include asking the individual to providerankings (e.g., “Not at all,” “Several days,” “More than half the days,”and “Nearly every day”) to questions about whether the individual hasbeen bothered in the past two weeks by i) little interest or pleasure indoing things; and ii) feeling down, depressed, or hopeless.

In some cases, executing an abatement protocol at block 312 can includescheduling a follow-up at block 322. Scheduling a follow-up can includeestablishing a time (e.g., a date and time of day) for a follow-up or aduration of time (e.g., a number of hours, days, or the like) that mustelapse before a follow-up should occur. In some cases, scheduling afollow-up at block 322 can be automatically performed using a presettime offset (e.g., automatically set for today's date+48 hours) or apreset value (e.g., automatically set for no earlier than 48 hours fromnow). In some cases, scheduling a follow-up at block 322 can includepresenting a prompt offering a number of preset options for a follow-up.In some cases, scheduling a follow-up at block 322 can includepresenting a prompt requesting the individual provide a time or aduration of time for the follow-up.

At block 324, the NLP system can present a follow-up message. Thefollow-up message can be presented based on the schedule defined atblock 322, or automatically. In some cases, presenting the follow-upmessage at block 324 can occur minutes, hours, days, or weeks afterexecution of the abatement protocol at block 312. Presenting thefollow-up message can include presenting a prompt seeking informationabout the individual, such as whether or not the crisis situation isongoing or how the individual managed the crisis situation. In somecases, presenting the follow-up message can include performing asubsequent PHQ and optionally comparing it to a PHQ performed at block320.

In some cases, when a confirmation response received at block 306 is adenial of the crisis situation, process 300 can end, permitting the NLPsystem to proceed with the normal workflow. However, in some cases, whena confirmation response received at block 306 is a denial of the crisissituation, the NLP system can present a denial prompt at 326. Presentingthe denial prompt can include presenting a prompt to elicit additionalinformation about why the detected crisis situation was not a crisissituation according to the individual. In some cases, presenting thedenial prompt at block 326 includes using the trigger phrase, such as todiscuss the trigger phrase at block 328. Discussing the trigger phrasecan include presenting prompts and/or comments to understand why thedetected crisis situation was not actually a crisis situation accordingto the individual. In some cases, presenting the denial prompt at block326 can include logging the response to the denial prompt. In somecases, the logged response to the denial prompt can be used to furtherupdate one or both classifiers, such as to inform a developer as to howto configure or train one or both of the classifiers to try and avoidtriggering when the individual is likely, certain, or nearly certain todeny that a crisis situation has occurred.

In some cases, when a confirmation response received at block 306 is adenial of the crisis situation, the NLP system can present the denialprompt at block 326, then proceed to nevertheless executing theabatement protocol at block 312 or a modified version thereof, such asexecuting a modified abatement protocol that takes into account that theindividual has denied the existence of a crisis situation. For example,such a modified abatement protocol may include actions that are designedto mitigate a crisis situation but not outwardly imply or acknowledgethe existence of a crisis situation (e.g., asking questions to help calmor ground an individual without specifically noting that a crisissituation is occurring).

Process 300 is depicted with a certain arrangement of blocks, however inother cases, these blocks can be performed in different orders, withadditional blocks, and/or some blocks removed. For example, in somecases, process 300 can proceed without presenting a confirmation promptor receiving a confirmation response, in which case the abatementprotocol can be executed at block 312 in response to receiving thetrigger signal at block 302. In another example, process 300 may endafter presentation of the confirmation prompt at block 304. In anotherexample, process 300 may include only blocks 302, 304, 306, 308, and310.

FIG. 4 is a flowchart depicting a process 400 for quantifying andidentifying a crisis situation according to certain aspects of thepresent disclosure. Process 400 can be performed by an NLP system in anysuitable environment, such as environment 100 of FIG. 1. Process 400 canbe repeated any number of times, such as once for each time the userprovides an input phrase.

At block 402, an input phrase is provided. Providing an input phrase atblock 402 can be similar to providing an input phrase at block 202 ofFIG. 2.

At block 404, a crisis score can be determined from the input phrase. Insome cases, determining a crisis score can include performing aclassification using one or both of a pattern matching classifier and atrained machine learning classifier. In some cases, the pattern matchingclassifier and trained machine learning classifier can be configured tooutput tiers or a numerical value associated with whether or not theinput phrase is likely to indicate a crisis situation. The output fromone or both classifiers can be used as a crisis score or a component fora crisis score (e.g., the component score from each classifier can beadded together and/or averaged for a total crisis score). In an example,a pattern matching classifier may output a component score by outputtinga score of 0 for no matches, a score of 1 for matches according to afirst set of matching criteria, and a score of 2 for matches accordingto a second set of matching criteria, and a score of 3 for matchesaccording to both the first set of matching criteria and the second setof matching criteria. Any number of scores can be used and any number orcombination of sets of matching criteria can be used. In anotherexample, the trained machine learning classifier can be configured andtrained to output scores (e.g., from 0 to 10) based on an increasinglikelihood that the input phrase indicates a crisis situation.

In some cases, process 400 can continue at block 406 by performing acrisis workflow if the crisis score determined at block 404 is outsideof a certain threshold. Being outside of a threshold can include beingbelow a threshold value, being above a threshold value, or being betweena lower threshold value and a higher threshold value. The threshold canbe a preset number, although that need not always be the case. In somecases, the threshold can be adjusted based on confirmation responsesfrom previous instances of the crisis workflow. For example, if athreshold is set at 5, but previous instance(s) of the crisis workflowbeing triggered due to a score of 6 have resulted in a confirmationresponse that is a denial of the crisis situation, the threshold may beautomatically or manually adjusted upwards, such as to 6 or 7. Otheradjustments can be made.

In some cases, instead of or in addition to block 406, the decisions toperform (e.g., trigger) a crisis workflow can be based on one or morehistorical crisis scores. At block 408, one or more historical crisisscores can be accessed. The historical crisis scores that are accessedcan be based on a number of scores (e.g., the previous 10 scores), aduration of time (e.g., any scores in the past 8 weeks), or otherwise(e.g., all scores or all scores with similar input phrases).

At block 410, a crisis score trend can be determined based on the crisisscore from block 404 and the historical crisis score(s) from block 408.In some cases, the crisis score trend can be an average of the crisisscore and historical crisis scores. In some cases, the crisis scoretrend can be a mathematical trend, such as a mathematical trend that canbe expressed as a function of sequence or time. For example, severalhistorical crisis scores and the crisis score may exhibit an upwardtrend (e.g., increasing 1-2 points per day or per sequential crisisscore). Such a mathematical trend can be determined through any suitabletechnique.

At block 412, the crisis workflow can be performed (e.g., triggered) ifthe crisis score trend fits one or more crisis-conditions. An example ofa crisis-condition can be a particular number of days or sequentialcrisis scores in sequence showing an increasing crisis score. In anotherexample, a crisis-condition can be a particular slope associated with acrisis score trend (e.g., a trend showing large consecutive increases incrisis score, even if the crisis score itself remains below a thresholdlevel). In some cases, the crisis-condition occurs when the crisis scoretrend indicates that a future crisis score (e.g., a subsequent crisisscore, a crisis score after a given period of time (e.g., a day), or acrisis score within a given period of time (e.g., a week)) will beoutside of a threshold.

Performing the crisis workflow at blocks 406, 412, or 414 can includeperforming any suitable crisis workflow, such as some or all of process300 of FIG. 3. In some cases, one or more trigger phrases thatcontributed to the crisis score can be provided to the crisis workflowas part of a trigger signal. In some cases, only one or more triggerphrases that contributed most to the crisis score will be provided.

Process 400 is depicted with a certain arrangement of blocks, however inother cases, these blocks can be performed in different orders, withadditional blocks, and/or some blocks removed. For example, in somecases, process 400 can proceed without block 406. In another example,process 400 can proceed without blocks 408, 410, 412, and 414.

FIG. 5 is a block diagram of an example system architecture 500 forimplementing features and processes of the present disclosure, such asthose presented with reference to processes 200, 300, and 400 of FIGS.2, 3, and 4, respectively. The architecture 500 can be used to implementa server (e.g., server 106 of FIG. 1), a user device (e.g., user device102 of FIG. 1), a communication device (e.g., communication device 112of FIG. 1), or any other suitable device for performing some or all ofthe aspects of the present disclosure. The architecture 500 can beimplemented on any electronic device that runs software applicationsderived from compiled instructions, including without limitationpersonal computers, servers, smart phones, electronic tablets, gameconsoles, email devices, and the like. In some implementations, thearchitecture 500 can include one or more processors 502, one or moreinput devices 504, one or more display devices 506, one or more networkinterfaces 508, and one or more computer-readable mediums 510. Each ofthese components can be coupled by bus 512.

Display device 506 can be any known display technology, including butnot limited to display devices using Liquid Crystal Display (LCD) orLight Emitting Diode (LED) technology. Processor(s) 502 can use anyknown processor technology, including but not limited to graphicsprocessors and multi-core processors. Input device 504 can be any knowninput device technology, including but not limited to a keyboard(including a virtual keyboard), mouse, track ball, and touch-sensitivepad or display. In some cases, audio inputs can be used to provide audiosignals, such as audio signals of an individual speaking. Bus 512 can beany known internal or external bus technology, including but not limitedto ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATA or FireWire.

Computer-readable medium 510 can be any medium that participates inproviding instructions to processor(s) 502 for execution, includingwithout limitation, non-volatile storage media (e.g., optical disks,magnetic disks, flash drives, etc.) or volatile media (e.g., SDRAM, ROM,etc.). The computer-readable medium (e.g., storage devices, mediums, andmemories) can include, for example, a cable or wireless signalcontaining a bit stream and the like. However, when mentioned,non-transitory computer-readable storage media expressly exclude mediasuch as energy, carrier signals, electromagnetic waves, and signals perse.

Computer-readable medium 510 can include various instructions forimplementing operating system 514 and applications 520 such as computerprograms. The operating system can be multi-user, multiprocessing,multitasking, multithreading, real-time and the like. The operatingsystem 514 performs basic tasks, including but not limited to:recognizing input from input device 504; sending output to displaydevice 506; keeping track of files and directories on computer-readablemedium 510; controlling peripheral devices (e.g., storage drives,interface devices, etc.) which can be controlled directly or through anI/O controller; and managing traffic on bus 512. Computer-readablemedium 510 can include various instructions for implementing firmwareprocesses, such as a BIOS. Computer-readable medium 510 can includevarious instructions for implementing any of the processes describedherein, including at least processes 200, 300, and 400 of FIGS. 2, 3,and 4, respectively.

Memory 518 can include high-speed random access memory and/ornon-volatile memory, such as one or more magnetic disk storage devices,one or more optical storage devices, and/or flash memory (e.g., NAND,NOR). The memory 518 (e.g., computer-readable storage devices, mediums,and memories) can include a cable or wireless signal containing a bitstream and the like. However, when mentioned, non-transitorycomputer-readable storage media expressly exclude media such as energy,carrier signals, electromagnetic waves, and signals per se. The memory518 can store an operating system, such as Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system such as VxWorks.

System controller 522 can be a service processor that operatesindependently of processor 502. In some implementations, systemcontroller 522 can be a baseboard management controller (BMC). Forexample, a BMC is a specialized service processor that monitors thephysical state of a computer, network server, or other hardware deviceusing sensors and communicating with the system administrator through anindependent connection. The BMC is configured on the motherboard or maincircuit board of the device to be monitored. The sensors of a BMC canmeasure internal physical variables such as temperature, humidity,power-supply voltage, fan speeds, communications parameters andoperating system (OS) functions.

The described features can be implemented advantageously in one or morecomputer programs that are executable on a programmable system includingat least one programmable processor coupled to receive data andinstructions from, and to transmit data and instructions to, a datastorage system, at least one input device, and at least one outputdevice. A computer program is a set of instructions that can be used,directly or indirectly, in a computer to perform a certain activity orbring about a certain result. A computer program can be written in anyform of programming language (e.g., Objective-C, Java), includingcompiled or interpreted languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors orcores, of any kind of computer. Generally, a processor will receiveinstructions and data from a read-only memory or a random access memoryor both. The essential elements of a computer are a processor forexecuting instructions and one or more memories for storing instructionsand data. Generally, a computer will also include, or be operativelycoupled to communicate with, one or more mass storage devices forstoring data files; such devices include magnetic disks, such asinternal hard disks and removable disks; magneto-optical disks; andoptical disks. Storage devices suitable for tangibly embodying computerprogram instructions and data include all forms of non-volatile memory,including by way of example semiconductor memory devices, such as EPROM,EEPROM, and flash memory devices; magnetic disks such as internal harddisks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, ASICs (application-specific integrated circuits).

To provide for interaction with a user, the features can be implementedon a computer having a display device such as a CRT (cathode ray tube)or LCD (liquid crystal display) monitor for displaying information tothe user and a keyboard and a pointing device such as a mouse or atrackball by which the user can provide input to the computer.

The features can be implemented in a computing system that includes aback-end component, such as a data server, or that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combinationthereof. The components of the system can be connected by any form ormedium of digital data communication such as a communication network.Examples of communication networks include, e.g., a LAN, a WAN, and thecomputers and networks forming the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

One or more features or steps of the disclosed embodiments can beimplemented using an application programming interface (API). An API candefine one or more parameters that are passed between a callingapplication and other software code (e.g., an operating system, libraryroutine, function) that provides a service, that provides data, or thatperforms an operation or a computation.

The API can be implemented as one or more calls in program code thatsend or receive one or more parameters through a parameter list or otherstructure based on a call convention defined in an API specificationdocument. A parameter can be a constant, a key, a data structure, anobject, an object class, a variable, a data type, a pointer, an array, alist, or another call. API calls and parameters can be implemented inany programming language. The programming language can define thevocabulary and calling convention that a programmer will employ toaccess functions supporting the API.

In some implementations, an API call can report to an application thecapabilities of a device running the application, such as inputcapability, output capability, processing capability, power capability,communications capability, and the like.

The foregoing description of the embodiments, including illustratedembodiments, has been presented only for the purpose of illustration anddescription and is not intended to be exhaustive or limiting to theprecise forms disclosed. Numerous modifications, adaptations, and usesthereof will be apparent to those skilled in the art. Numerous changesto the disclosed embodiments can be made in accordance with thedisclosure herein, without departing from the spirit or scope of theinvention. Thus, the breadth and scope of the present invention shouldnot be limited by any of the above described embodiments.

Although the invention has been illustrated and described with respectto one or more implementations, equivalent alterations and modificationswill occur or be known to others skilled in the art upon the reading andunderstanding of this specification and the annexed drawings. Inaddition, while a particular feature of the invention may have beendisclosed with respect to only one of several implementations, suchfeature may be combined with one or more other features of the otherimplementations as may be desired and advantageous for any given orparticular application.

The terminology used herein is for the purpose of describing particularembodiments only, and is not intended to be limiting of the invention.As used herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. Furthermore, to the extent that the terms “including,”“includes,” “having,” “has,” “with,” or variants thereof, are used ineither the detailed description and/or the claims, such terms areintended to be inclusive in a manner similar to the term “comprising.”

As used below, any reference to a series of examples is to be understoodas a reference to each of those examples disjunctively (e.g., “Examples1-4” is to be understood as “Examples 1, 2, 3, or 4”).

Example 1 is a system, comprising: one or more data processors; and anon-transitory computer-readable storage medium containing instructionswhich, when executed on the one or more data processors, cause the oneor more data processors to perform operations including: receiving aninput phrase associated with a crisis situation; applying a firstclassifier on the input phrase to determine that the input phrase doesnot trigger the first classifier; applying a second classifier on theinput phrase to determine that the input phrase triggers the secondclassifier, wherein the first classifier is one of a pattern matchingclassifier and a trained machine learning classifier, and wherein thesecond classifier is the other of the pattern matching classifier andthe trained machine learning classifier; generating a trigger signal inresponse to determining that the input phrase triggers the secondclassifier; and executing an abatement protocol in response togeneration of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.

Example 2 is the system of example(s) 1, wherein executing the abatementprotocol includes i) accessing and presenting a text string containingemergency contact information, ii) accessing and presenting a linkconfigured to facilitate an emergency contact connection upon actuation;iii) sending a signal to automatically facilitate an emergency contactconnection; iv) initiating a patient health questionnaire protocol; v)generating a prompt for selecting a crisis management tool; vi)automatically initiating a crisis management tool; or vii) anycombination of i-vi.

Example 3 is the system of example(s) 1 or 2, wherein the operationsfurther include: presenting a crisis confirmation prompt in response todetermining that the input phrase triggers the second classifier; andreceiving a confirmation response in response to presenting theconfirmation prompt, the confirmation response including a confirmationof the crisis situation or a denial of the crisis situation, whereinexecuting the abatement protocol occurs in response to receiving theconfirmation of the crisis situation.

Example 4 is the system of example(s) 3, wherein the operations furtherinclude: identifying a trigger phrase in response to determining thatthe input phrase triggers the second classifier, wherein the triggerphrase includes a portion of the input phrase that matched a regularexpression when the second classifier is the pattern matchingclassifier, and wherein the trigger phrase includes the input phrasewhen the second classifier is the trained machine learning classifier;and presenting a denial prompt in response to receiving the denial ofthe crisis situation, wherein presenting the denial prompt includesusing the trigger phrase.

Example 5 is the system of example(s) 4, wherein the operations furtherinclude: logging the confirmation response in association with thetrigger phrase; and updating at least one of the first classifier andthe second classifier using the logged confirmation response and thetrigger phrase.

Example 6 is the system of example(s) 4 or 5, wherein executing theabatement protocol includes scheduling a follow-up contact for a futuretime, wherein the operations further include presenting a follow-upmessage at the future time, and wherein presenting the follow-up messageincludes presenting the trigger phrase.

Example 7 is the system of example(s) 1-6, wherein the operationsfurther include determining a crisis score using the input phrase,wherein generating the trigger signal is further based on adetermination that the crisis score is outside of a threshold range.

Example 8 is the system of example(s) 1-7, wherein the operation furtherinclude: determining a crisis score using the input phrase; accessingone or more historical crisis scores; determining a crisis score trendusing the crisis score and the one or more historical crisis scores; anddetermining a future crisis score using the crisis score and the crisisscore trend, wherein generating the trigger signal is further based on adetermination that the future crisis score is outside of a thresholdrange.

Example 9 is the system of example(s) 1-8, wherein the first classifieris the pattern matching classifier and the second classifier is thetrained machine learning classifier.

Example 10 is the system of example(s) 1-9, wherein the trained machinelearning classifier is trained using a set of training input phrasesthat were determined to not trigger the pattern matching classifier.

Example 11 is the system of example(s) 1-10, wherein the trained machinelearning classifier is trained using a set of training input phrasesincluding a first subset of training input phrases associated withcrisis situations and a second subset of training input phrases notassociated with crisis situations.

Example 12 is a computer-implemented method, comprising: receiving aninput phrase associated with a crisis situation; applying a firstclassifier on the input phrase to determine that the input phrase doesnot trigger the first classifier; applying a second classifier on theinput phrase to determine that the input phrase triggers the secondclassifier, wherein the first classifier is one of a pattern matchingclassifier and a trained machine learning classifier, and wherein thesecond classifier is the other of the pattern matching classifier andthe trained machine learning classifier; generating a trigger signal inresponse to determining that the input phrase triggers the secondclassifier; and executing an abatement protocol in response togeneration of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.

Example 13 is the computer-implemented method of example(s) 12, whereinexecuting the abatement protocol includes i) accessing and presenting atext string containing emergency contact information, ii) accessing andpresenting a link configured to facilitate an emergency contactconnection upon actuation; iii) sending a signal to automaticallyfacilitate an emergency contact connection; iv) initiating a patienthealth questionnaire protocol; v) generating a prompt for selecting acrisis management tool; vi) automatically initiating a crisis managementtool; or vii) any combination of i-vi.

Example 14 is the computer-implemented method of example(s) 12 or 13,further comprising: presenting a crisis confirmation prompt in responseto determining that the input phrase triggers the second classifier; andreceiving a confirmation response in response to presenting theconfirmation prompt, the confirmation response including a confirmationof the crisis situation or a denial of the crisis situation, whereinexecuting the abatement protocol occurs in response to receiving theconfirmation of the crisis situation.

Example 15 is the computer-implemented method of example(s) 14, furthercomprising: identifying a trigger phrase in response to determining thatthe input phrase triggers the second classifier, wherein the triggerphrase includes a portion of the input phrase that matched a regularexpression when the second classifier is the pattern matchingclassifier, and wherein the trigger phrase includes the input phrasewhen the second classifier is the trained machine learning classifier;and presenting a denial prompt in response to receiving the denial ofthe crisis situation, wherein presenting the denial prompt includesusing the trigger phrase.

Example 16 is the computer-implemented method of example(s) 15, furthercomprising: logging the confirmation response in association with thetrigger phrase; and updating at least one of the first classifier andthe second classifier using the logged confirmation response and thetrigger phrase.

Example 17 is the computer-implemented method of example(s) 15 or 16,wherein executing the abatement protocol includes scheduling a follow-upcontact for a future time, wherein the method further comprisespresenting a follow-up message at the future time, and whereinpresenting the follow-up message includes presenting the trigger phrase.

Example 18 is the computer-implemented method of example(s) 12-17,further comprising determining a crisis score using the input phrase,wherein generating the trigger signal is further based on adetermination that the crisis score is outside of a threshold range.

Example 19 is the computer-implemented method of example(s) 12-18,further comprising: determining a crisis score using the input phrase;accessing one or more historical crisis scores; determining a crisisscore trend using the crisis score and the one or more historical crisisscores; and determining a future crisis score using the crisis score andthe crisis score trend, wherein generating the trigger signal is furtherbased on a determination that the future crisis score is outside of athreshold range.

Example 20 is the computer-implemented method of example(s) 12-19,wherein the first classifier is the pattern matching classifier and thesecond classifier is the trained machine learning classifier.

Example 21 is the computer-implemented method of example(s) 12-20,wherein the trained machine learning classifier is trained using a setof training input phrases that were determined to not trigger thepattern matching classifier.

Example 22 is the computer-implemented method of example(s) 12-21,wherein the trained machine learning classifier is trained using a setof training input phrases including a first subset of training inputphrases associated with crisis situations and a second subset oftraining input phrases not associated with crisis situations.

Example 23 is a computer-program product tangibly embodied in anon-transitory machine-readable storage medium, including instructionsconfigured to cause a data processing apparatus to perform operationsincluding: receiving an input phrase associated with a crisis situation;applying a first classifier on the input phrase to determine that theinput phrase does not trigger the first classifier; applying a secondclassifier on the input phrase to determine that the input phrasetriggers the second classifier, wherein the first classifier is one of apattern matching classifier and a trained machine learning classifier,and wherein the second classifier is the other of the pattern matchingclassifier and the trained machine learning classifier; generating atrigger signal in response to determining that the input phrase triggersthe second classifier; and executing an abatement protocol in responseto generation of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.

Example 24 is the computer-program product of example(s) 23, whereinexecuting the abatement protocol includes i) accessing and presenting atext string containing emergency contact information, ii) accessing andpresenting a link configured to facilitate an emergency contactconnection upon actuation; iii) sending a signal to automaticallyfacilitate an emergency contact connection; iv) initiating a patienthealth questionnaire protocol; v) generating a prompt for selecting acrisis management tool; vi) automatically initiating a crisis managementtool; or vii) any combination of i-vi.

Example 25 is the computer-program product of example(s) 23 or 24,wherein the operations further include: presenting a crisis confirmationprompt in response to determining that the input phrase triggers thesecond classifier; and receiving a confirmation response in response topresenting the confirmation prompt, the confirmation response includinga confirmation of the crisis situation or a denial of the crisissituation, wherein executing the abatement protocol occurs in responseto receiving the confirmation of the crisis situation.

Example 26 is the computer-program product of example(s) 25, wherein theoperations further include: identifying a trigger phrase in response todetermining that the input phrase triggers the second classifier,wherein the trigger phrase includes a portion of the input phrase thatmatched a regular expression when the second classifier is the patternmatching classifier, and wherein the trigger phrase includes the inputphrase when the second classifier is the trained machine learningclassifier; and presenting a denial prompt in response to receiving thedenial of the crisis situation, wherein presenting the denial promptincludes using the trigger phrase.

Example 27 is the computer-program product of example(s) 26, wherein theoperations further include: logging the confirmation response inassociation with the trigger phrase; and updating at least one of thefirst classifier and the second classifier using the logged confirmationresponse and the trigger phrase.

Example 28 is the computer-program product of example(s) 26 or 27,wherein executing the abatement protocol includes scheduling a follow-upcontact for a future time, wherein the operations further includepresenting a follow-up message at the future time, and whereinpresenting the follow-up message includes presenting the trigger phrase.

Example 29 is the computer-program product of example(s) 23-28, whereinthe operations further include determining a crisis score using theinput phrase, wherein generating the trigger signal is further based ona determination that the crisis score is outside of a threshold range.

Example 30 is the computer-program product of example(s) 23-29, whereinthe operation further include: determining a crisis score using theinput phrase; accessing one or more historical crisis scores;determining a crisis score trend using the crisis score and the one ormore historical crisis scores; and determining a future crisis scoreusing the crisis score and the crisis score trend, wherein generatingthe trigger signal is further based on a determination that the futurecrisis score is outside of a threshold range.

Example 31 is the computer-program product of example(s) 23-29, whereinthe first classifier is the pattern matching classifier and the secondclassifier is the trained machine learning classifier.

Example 32 is the computer-program product of example(s) 23-31, whereinthe trained machine learning classifier is trained using a set oftraining input phrases that were determined to not trigger the patternmatching classifier.

Example 33 is the computer-program product of example(s) 23-32, whereinthe trained machine learning classifier is trained using a set oftraining input phrases including a first subset of training inputphrases associated with crisis situations and a second subset oftraining input phrases not associated with crisis situations.

What is claimed is:
 1. A system, comprising: one or more dataprocessors; and a non-transitory computer-readable storage mediumcontaining instructions which, when executed on the one or more dataprocessors, cause the one or more data processors to perform operationsincluding: receiving an input phrase associated with a crisis situation;applying a first classifier on the input phrase to determine that theinput phrase does not trigger the first classifier; applying a secondclassifier on the input phrase to determine that the input phrasetriggers the second classifier, wherein the first classifier is one of apattern matching classifier and a trained machine learning classifier,and wherein the second classifier is the other of the pattern matchingclassifier and the trained machine learning classifier; generating atrigger signal in response to determining that the input phrase triggersthe second classifier; and executing an abatement protocol in responseto generation of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.
 2. The system ofclaim 1, wherein executing the abatement protocol includes i) accessingand presenting a text string containing emergency contact information,ii) accessing and presenting a link configured to begin initiation of anemergency contact connection upon actuation; iii) sending a signal toautomatically begin initiation of an emergency contact connection; iv)initiating a patient health questionnaire protocol; v) generating aprompt for selecting a crisis management tool; vi) automaticallyinitiating a crisis management tool; or vii) any combination of i-vi. 3.The system of claim 1, wherein the operations further include:presenting a crisis confirmation prompt in response to determining thatthe input phrase triggers the second classifier; and receiving aconfirmation response in response to presenting the confirmation prompt,the confirmation response including a confirmation of the crisissituation or a denial of the crisis situation, wherein executing theabatement protocol occurs in response to receiving the confirmation ofthe crisis situation.
 4. The system of claim 3, wherein the operationsfurther include: identifying a trigger phrase in response to determiningthat the input phrase triggers the second classifier, wherein thetrigger phrase includes a portion of the input phrase that matched aregular expression when the second classifier is the pattern matchingclassifier, and wherein the trigger phrase includes the input phrasewhen the second classifier is the trained machine learning classifier;and presenting a denial prompt in response to receiving the denial ofthe crisis situation, wherein presenting the denial prompt includesusing the trigger phrase.
 5. The system of claim 4, wherein theoperations further include: logging the confirmation response inassociation with the trigger phrase; and updating at least one of thefirst classifier and the second classifier using the logged confirmationresponse and the trigger phrase.
 6. The system of claim 4, whereinexecuting the abatement protocol includes scheduling a follow-up contactfor a future time, wherein the operations further include presenting afollow-up message at the future time, and wherein presenting thefollow-up message includes presenting the trigger phrase.
 7. The systemof claim 1, wherein the operations further include determining a crisisscore using the input phrase, wherein generating the trigger signal isfurther based on a determination that the crisis score is outside of athreshold range.
 8. The system of claim 1, wherein the operation furtherinclude: determining a crisis score using the input phrase; accessingone or more historical crisis scores; determining a crisis score trendusing the crisis score and the one or more historical crisis scores; anddetermining a future crisis score using the crisis score and the crisisscore trend, wherein generating the trigger signal is further based on adetermination that the future crisis score is outside of a thresholdrange.
 9. The system of claim 1, wherein the first classifier is thepattern matching classifier and the second classifier is the trainedmachine learning classifier.
 10. The system of claim 9, wherein thetrained machine learning classifier is trained using a set of traininginput phrases that were determined to not trigger the pattern matchingclassifier.
 11. The system of claim 1, wherein the trained machinelearning classifier is trained using a set of training input phrasesincluding a first subset of training input phrases associated withcrisis situations and a second subset of training input phrases notassociated with crisis situations.
 12. A computer-implemented method,comprising: receiving an input phrase associated with a crisissituation; applying a first classifier on the input phrase to determinethat the input phrase does not trigger the first classifier; applying asecond classifier on the input phrase to determine that the input phrasetriggers the second classifier, wherein the first classifier is one of apattern matching classifier and a trained machine learning classifier,and wherein the second classifier is the other of the pattern matchingclassifier and the trained machine learning classifier; generating atrigger signal in response to determining that the input phrase triggersthe second classifier; and executing an abatement protocol in responseto generation of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.
 13. Thecomputer-implemented method of claim 12, wherein executing the abatementprotocol includes i) accessing and presenting a text string containingemergency contact information, ii) accessing and presenting a linkconfigured to facilitate an emergency contact connection upon actuation;iii) sending a signal to automatically facilitate an emergency contactconnection; iv) initiating a patient health questionnaire protocol; v)generating a prompt for selecting a crisis management tool; vi)automatically initiating a crisis management tool; or vii) anycombination of i-vi.
 14. The computer-implemented method of claim 12,further comprising: presenting a crisis confirmation prompt in responseto determining that the input phrase triggers the second classifier; andreceiving a confirmation response in response to presenting theconfirmation prompt, the confirmation response including a confirmationof the crisis situation or a denial of the crisis situation, whereinexecuting the abatement protocol occurs in response to receiving theconfirmation of the crisis situation.
 15. The computer-implementedmethod of claim 14, further comprising: identifying a trigger phrase inresponse to determining that the input phrase triggers the secondclassifier, wherein the trigger phrase includes a portion of the inputphrase that matched a regular expression when the second classifier isthe pattern matching classifier, and wherein the trigger phrase includesthe input phrase when the second classifier is the trained machinelearning classifier; and presenting a denial prompt in response toreceiving the denial of the crisis situation, wherein presenting thedenial prompt includes using the trigger phrase.
 16. Thecomputer-implemented method of claim 15, further comprising: logging theconfirmation response in association with the trigger phrase; andupdating at least one of the first classifier and the second classifierusing the logged confirmation response and the trigger phrase.
 17. Thecomputer-implemented method of claim 15, wherein executing the abatementprotocol includes scheduling a follow-up contact for a future time,wherein the method further comprises presenting a follow-up message atthe future time, and wherein presenting the follow-up message includespresenting the trigger phrase.
 18. The computer-implemented method ofclaim 12, further comprising determining a crisis score using the inputphrase, wherein generating the trigger signal is further based on adetermination that the crisis score is outside of a threshold range. 19.The computer-implemented method of claim 12, further comprising:determining a crisis score using the input phrase; accessing one or morehistorical crisis scores; determining a crisis score trend using thecrisis score and the one or more historical crisis scores; anddetermining a future crisis score using the crisis score and the crisisscore trend, wherein generating the trigger signal is further based on adetermination that the future crisis score is outside of a thresholdrange.
 20. The computer-implemented method of claim 12, wherein thefirst classifier is the pattern matching classifier and the secondclassifier is the trained machine learning classifier.
 21. Thecomputer-implemented method of claim 20, wherein the trained machinelearning classifier is trained using a set of training input phrasesthat were determined to not trigger the pattern matching classifier. 22.A computer-program product tangibly embodied in a non-transitorymachine-readable storage medium, including instructions configured tocause a data processing apparatus to perform operations including:receiving an input phrase associated with a crisis situation; applying afirst classifier on the input phrase to determine that the input phrasedoes not trigger the first classifier; applying a second classifier onthe input phrase to determine that the input phrase triggers the secondclassifier, wherein the first classifier is one of pattern matchingclassifier and a trained machine learning classifier, and wherein thesecond classifier is the other of the pattern matching classifier andthe trained machine learning classifier; generating a trigger signal inresponse to determining that the input phrase triggers the secondclassifier; and executing an abatement protocol in response togeneration of the trigger signal, wherein execution of the abatementprotocol is intended to mitigate the crisis situation.
 23. Thecomputer-program product of claim 22, wherein executing the abatementprotocol includes i) accessing and presenting a text string containingemergency contact information, ii) accessing and presenting a linkconfigured to facilitate an emergency contact connection upon actuation;iii) sending a signal to automatically facilitate an emergency contactconnection; iv) initiating a patient health questionnaire protocol; v)generating a prompt for selecting a crisis management tool; vi)automatically initiating a crisis management tool; or vii) anycombination of i-vi.
 24. The computer-program product of claim 22,wherein the operations further include: presenting a crisis confirmationprompt in response to determining that the input phrase triggers thesecond classifier; and receiving a confirmation response in response topresenting the confirmation prompt, the confirmation response includinga confirmation of the crisis situation or a denial of the crisissituation, wherein executing the abatement protocol occurs in responseto receiving the confirmation of the crisis situation.
 25. Thecomputer-program product of claim 24, wherein the operations furtherinclude: identifying a trigger phrase in response to determining thatthe input phrase triggers the second classifier, wherein the triggerphrase includes a portion of the input phrase that matched a regularexpression when the second classifier is the pattern matchingclassifier, and wherein the trigger phrase includes the input phrasewhen the second classifier is the trained machine learning classifier;and presenting a denial prompt in response to receiving the denial ofthe crisis situation, wherein presenting the denial prompt includesusing the trigger phrase.
 26. The computer-program product of claim 25,wherein the operations further include: logging the confirmationresponse in association with the trigger phrase; and updating at leastone of the first classifier and the second classifier using the loggedconfirmation response and the trigger phrase.
 27. The computer-programproduct of claim 25, wherein executing the abatement protocol includesscheduling a follow-up contact for a future time, wherein the operationsfurther include presenting a follow-up message at the future time, andwherein presenting the follow-up message includes presenting the triggerphrase.
 28. The computer-program product of claim 22, wherein theoperation further include: determining a crisis score using the inputphrase; accessing one or more historical crisis scores; determining acrisis score trend using the crisis score and the one or more historicalcrisis scores; and determining a future crisis score using the crisisscore and the crisis score trend, wherein generating the trigger signalis further based on a determination that the future crisis score isoutside of a threshold range.
 29. The computer-program product of claim22, wherein the first classifier is the pattern matching classifier andthe second classifier is the trained machine learning classifier. 30.The computer-program product of claim 29, wherein the trained machinelearning classifier is trained using a set of training input phrasesthat were determined to not trigger the pattern matching classifier.